1 |
A multi-scale investigation of the human communication system's response to visual disruption
|
|
|
|
In: R Soc Open Sci (2022)
|
|
Abstract:
In human communication, when the speech is disrupted, the visual channel (e.g. manual gestures) can compensate to ensure successful communication. Whether speech also compensates when the visual channel is disrupted is an open question, and one that significantly bears on the status of the gestural modality. We test whether gesture and speech are dynamically co-adapted to meet communicative needs. To this end, we parametrically reduce visibility during casual conversational interaction and measure the effects on speakers' communicative behaviour using motion tracking and manual annotation for kinematic and acoustic analyses. We found that visual signalling effort was flexibly adapted in response to a decrease in visual quality (especially motion energy, gesture rate, size, velocity and hold-time). Interestingly, speech was also affected: speech intensity increased in response to reduced visual quality (particularly in speech-gesture utterances, but independently of kinematics). Our findings highlight that multi-modal communicative behaviours are flexibly adapted at multiple scales of measurement and question the notion that gesture plays an inferior role to speech.
|
|
Keyword:
Psychology and Cognitive Neuroscience
|
|
URL: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC9006025/ https://doi.org/10.1098/rsos.211489
|
|
BASE
|
|
Hide details
|
|
2 |
Competition Reduces Response Times in Multiparty Conversation
|
|
|
|
BASE
|
|
Show details
|
|
3 |
Speakers exhibit a multimodal Lombard effect in noise
|
|
|
|
In: Sci Rep (2021)
|
|
BASE
|
|
Show details
|
|
4 |
Competition Reduces Response Times in Multiparty Conversation
|
|
|
|
In: Front Psychol (2021)
|
|
BASE
|
|
Show details
|
|
6 |
Hand gestures have predictive potential during face-to-face conversation ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
Hand gestures have predictive potential during face-to-face conversation ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Hand gestures have predictive potential during face-to-face conversation ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
How visual cues to speech rate influence speech perception ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
How visual cues to speech rate influence speech perception ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
QJE-STD-19-052.R2-Supplementary_Material – Supplemental material for How visual cues to speech rate influence speech perception ...
|
|
|
|
BASE
|
|
Show details
|
|
13 |
QJE-STD-19-052.R2-Supplementary_Material – Supplemental material for How visual cues to speech rate influence speech perception ...
|
|
|
|
BASE
|
|
Show details
|
|
14 |
Multimodality and the origin of a novel communication system in face-to-face interaction
|
|
|
|
BASE
|
|
Show details
|
|
15 |
Evidence for children’s online integration of simultaneous information from speech and iconic gestures: an ERP study
|
|
|
|
BASE
|
|
Show details
|
|
16 |
Multimodality and the origin of a novel communication system in face-to-face interaction
|
|
|
|
BASE
|
|
Show details
|
|
17 |
Supplementary material from "Multimodality and the origin of a novel communication system in face-to-face interaction" ...
|
|
|
|
BASE
|
|
Show details
|
|
18 |
Supplementary material from "Multimodality and the origin of a novel communication system in face-to-face interaction" ...
|
|
|
|
BASE
|
|
Show details
|
|
19 |
See the attached file for 6 titles from Multimodality and the origin of a novel communication system in face-to-face interaction ...
|
|
|
|
BASE
|
|
Show details
|
|
20 |
See the attached file for 6 titles from Multimodality and the origin of a novel communication system in face-to-face interaction ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|